63 research outputs found
Towards Spatio-Temporal Aware Traffic Time Series Forecasting--Full Version
Traffic time series forecasting is challenging due to complex spatio-temporal
dynamics time series from different locations often have distinct patterns; and
for the same time series, patterns may vary across time, where, for example,
there exist certain periods across a day showing stronger temporal
correlations. Although recent forecasting models, in particular deep learning
based models, show promising results, they suffer from being spatio-temporal
agnostic. Such spatio-temporal agnostic models employ a shared parameter space
irrespective of the time series locations and the time periods and they assume
that the temporal patterns are similar across locations and do not evolve
across time, which may not always hold, thus leading to sub-optimal results. In
this work, we propose a framework that aims at turning spatio-temporal agnostic
models to spatio-temporal aware models. To do so, we encode time series from
different locations into stochastic variables, from which we generate
location-specific and time-varying model parameters to better capture the
spatio-temporal dynamics. We show how to integrate the framework with canonical
attentions to enable spatio-temporal aware attentions. Next, to compensate for
the additional overhead introduced by the spatio-temporal aware model parameter
generation process, we propose a novel window attention scheme, which helps
reduce the complexity from quadratic to linear, making spatio-temporal aware
attentions also have competitive efficiency. We show strong empirical evidence
on four traffic time series datasets, where the proposed spatio-temporal aware
attentions outperform state-of-the-art methods in term of accuracy and
efficiency. This is an extended version of "Towards Spatio-Temporal Aware
Traffic Time Series Forecasting", to appear in ICDE 2022 [1], including
additional experimental results.Comment: Accepted at ICDE 202
Unsupervised Path Representation Learning with Curriculum Negative Sampling
Path representations are critical in a variety of transportation
applications, such as estimating path ranking in path recommendation systems
and estimating path travel time in navigation systems. Existing studies often
learn task-specific path representations in a supervised manner, which require
a large amount of labeled training data and generalize poorly to other tasks.
We propose an unsupervised learning framework Path InfoMax (PIM) to learn
generic path representations that work for different downstream tasks. We first
propose a curriculum negative sampling method, for each input path, to generate
a small amount of negative paths, by following the principles of curriculum
learning. Next, \emph{PIM} employs mutual information maximization to learn
path representations from both a global and a local view. In the global view,
PIM distinguishes the representations of the input paths from those of the
negative paths. In the local view, \emph{PIM} distinguishes the input path
representations from the representations of the nodes that appear only in the
negative paths. This enables the learned path representations to encode both
global and local information at different scales. Extensive experiments on two
downstream tasks, ranking score estimation and travel time estimation, using
two road network datasets suggest that PIM significantly outperforms other
unsupervised methods and is also able to be used as a pre-training method to
enhance supervised path representation learning.Comment: This paper has been accepted by IJCAI-2
Triformer:Triangular, Variable-Specific Attentions for Long Sequence Multivariate Time Series Forecasting--Full Version
A variety of real-world applications rely on far future information to make
decisions, thus calling for efficient and accurate long sequence multivariate
time series forecasting. While recent attention-based forecasting models show
strong abilities in capturing long-term dependencies, they still suffer from
two key limitations. First, canonical self attention has a quadratic complexity
w.r.t. the input time series length, thus falling short in efficiency. Second,
different variables' time series often have distinct temporal dynamics, which
existing studies fail to capture, as they use the same model parameter space,
e.g., projection matrices, for all variables' time series, thus falling short
in accuracy. To ensure high efficiency and accuracy, we propose Triformer, a
triangular, variable-specific attention. (i) Linear complexity: we introduce a
novel patch attention with linear complexity. When stacking multiple layers of
the patch attentions, a triangular structure is proposed such that the layer
sizes shrink exponentially, thus maintaining linear complexity. (ii)
Variable-specific parameters: we propose a light-weight method to enable
distinct sets of model parameters for different variables' time series to
enhance accuracy without compromising efficiency and memory usage. Strong
empirical evidence on four datasets from multiple domains justifies our design
choices, and it demonstrates that Triformer outperforms state-of-the-art
methods w.r.t. both accuracy and efficiency. This is an extended version of
"Triformer: Triangular, Variable-Specific Attentions for Long Sequence
Multivariate Time Series Forecasting", to appear in IJCAI 2022 [Cirstea et al.,
2022a], including additional experimental results
- …